Analysis notebook guidelines

Jupyter notebooks can be a very useful tool for data analysis and measurement logging. Some of the main benefits are

  • Direct access to data and analysis code. The analysis is therefore reproducible.

  • Conversion of analysis notebooks into a website (ask Serwan for details).

  • Can be combined with version control, allowing tracking of changes.

For everyone using SilQ/QCoDeS, the first point is especially true. Although Jupyter Notebook has several advantages when used as an analysis/logging tool, difficulties can arise. Speaking from personal experiences, the main struggle was how to organize the Jupyter Notebooks. There is a fine balance between having too many notebooks, and having too few notebooks that explode in size. Below we describe guidelines that we have found gave us good results. An added benefit of this structure is that it can be straightforwardly converted to a website.

Structure of an analysis notebook

Part 0. Initialization cell

Each analysis notebook typically starts with a single initialization cell containing some code that loads all of the necessary packages/scripts

Part 1. Summary

Next we have a header with caption Summary.
The summary should give a description of everything done in the notebook, and all the relevant findings/issues.
The idea is that later on, you can get a complete idea of everything measured in the notebook without having to look into the body of the notebook.
The first cell should be a markdown cell where the first paragraph is a summary of the most important points of the notebook.
It is important to keep this as a single paragraph because it is extracted for the website.
When referring to a measurement in the summary, a hyperlink to the relevant measurement (sub)header should be used (see bottom for info on hyperlinks).
The first summary cell can also contain additional paragraphs, which should be used to expand on topics briefly described in the first paragraph, and to note other small findings not worthy of the first paragraph.
A good example is an issue you were facing and how you managed to solve it if at all.
Don’t worry about making these paragraphs lengthy.
After the first summary cell, subheaders should be used in which the most important data of the notebook should be displayed.
It is okay to copy code from other parts of the notebook, just be sure that the cell in the summary runs without having to execute cells further ahead.
Some effort should be put into making these figures clear (and pretty), as it should immediately clarify what the findings are.
These subheaders are also useful for joining data from multiple headers in the body of the analysis notebook, and performing some (minor) analysis on them.
As a general rule, measurements referred in the first paragraph should be displayed here.

Part 2. Measurements

After the summary, the actual measurements are collected and analysed.
All measurements should be loaded chronologically, and grouped in a hierarchical structure.
The disadvantage of a chronological order is that switching between measurements will also be reflected in the analysis notebook, which may then lack a clear storyline.

Hierarchical structuring of measurements

Measurements should be grouped together using headers, subheaders, etc.
It is up to the person performing the analysis to decide at what level measurements should be grouped together.
The header level should generally be per topic.
Try to aim for notebooks containing ~4-10 headers, though it may of course vary per measurement set.
If you notice you need more than 10 headers, it may make sense to think if you can split the notebook into multiple notebooks.
Each header, subheader, etc. should have a title, followed by a line briefly summarizing the result.
Only add the most important information, and if possible, prepend with a green success or red fail specifying if the measurement(s) was successful. The following HTML can be used in markdown cells to create a success/fail signifier. - **<span style="color:forestgreen">Success</span>** - **<span style="color:red">fail</span>**
Any more information should be placed in a markdown cell below the header.
After this, usually data is loaded and displayed, and text is added to describe the data and draw conclusions from it.

# Guidelines for analysis notebooks

  • Try to avoid multiple copies of analysis blocks, especially if it’s long. Instead, create a function and place it in a script folder.

  • When creating a subsection, always use one level lower (i.e. not straight from header to subsubheader)

  • For the summary, try to create a storyline. Things don’t necessarily have to be explained chronologically.

  • When the notebook is finished, ensure that it can run from start to finish by pressing Kernel -> Restart and run all

  • Do not write code cells that only output values. For example. if you want to output a frequency from a fit, perform a print statement with some text explaining what is being output (and units!)

  • To create a hyperlink, first add an HTML tag at the destination markup cell (typically a header).

  • The HTML anchor tag must be written as: <a id='your_tag_here'></a>

  • To create a hyperlink in a markdown cell, simply include “[text with hyperlink](#your_tag_here)”, with result: text with hyperlink.